Segment-based speaker verification using dynamically generated phrases

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for verifying an identity of a user. The methods, systems, and apparatus include actions of receiving a request for a verification phrase for verifying an identity of a user. Additional actions include, in response to receiving the request for the verification phrase for verifying the identity of the user, identifying subwords to be included in the verification phrase and in response to identifying the subwords to be included in the verification phrase, obtaining a candidate phrase that includes at least some of the identified subwords as the verification phrase. Further actions include providing the verification phrase as a response to the request for the verification phrase for verifying the identity of the user.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.14/242,098, filed Apr. 1, 2014, the contents of which is incorporated byreference.

TECHNICAL FIELD

This disclosure generally relates to speaker verification.

BACKGROUND

A computer may perform speaker verification to verify an identity of aspeaker. For example, a computer may verify an identity of speaker as aparticular user based on verifying that acoustic data representing thespeaker's voice matches acoustic data representing the particular user'svoice.

SUMMARY

In general, an aspect of the subject matter described in thisspecification may involve a process for verifying an identity of aspeaker. Speaker verification occurs by matching acoustic datarepresenting an utterance from a speaker with acoustic data representingutterances from the particular user.

The system may perform speaker verification by always asking a speakerto speak the same phrase, e.g., “FIXED VERIFICATION PHRASE.” Thisapproach may be accurate but may be prone to spoofing. For example, arecording of the particular user speaking the phrase may be replayed.Alternatively, the system may allow a speaker to independently speak aphrase, e.g., “RANDOM VERIFICATION PHRASE.” However, this approach maybe less accurate. For example, the system may be unable to determinewhat phrase was said by the speaker.

The system may address the above issues with speaker verification byproviding a verification phrase that is dynamically generated based ontraining acoustic data stored for the particular user. For example, thesystem may provide the verification phrase, e.g., “HAMMER,” to a speakerto be verified as a particular user based on determining that the systemstores training acoustic data representing the particular user speakingthe subword “HAM.”

In response to providing the verification phrase, the system may obtainacoustic data representing the speaker speaking the verification phraseand verify an identity of the speaker as the particular user using theobtained acoustic data. For example, the system may verify the identityof the speaker as the particular user based on determining that obtainedacoustic data representing the speaker speaking the subword “HAM” in“HAMMER” matches training acoustic data representing the particular userspeaking the subword “HAM.”

If the system verifies an identity of the speaker as the particularuser, the system may store the obtained acoustic data as trainingacoustic data for the particular user. For example, the system may storeacoustic data representing the speaker speaking the subword “MER” asacoustic data representing the particular user speaking the subword“MER.” In the future when verifying an identity of a speaker as theparticular user, the system may compare acoustic data representing aspeaker speaking the subword “MER” with the newly stored trainingacoustic data representing the particular user speaking the subword“MER.” For example, the next time the system performs speakerverification to verify a speaker as the particular user, the system mayprovide a different verification phrase, e.g., “JAMMER,” based ondetermining that the system stores training acoustic data representingthe particular user speaking the subword “MER.”

In some aspects, the subject matter described in this specification maybe embodied in methods that may include the actions of receiving arequest for a verification phrase for verifying an identity of a user.Additional actions include, in response to receiving the request for theverification phrase for verifying the identity of the user, identifyingsubwords to be included in the verification phrase and in response toidentifying the subwords to be included in the verification phrase,obtaining a candidate phrase that includes at least some of theidentified subwords as the verification phrase. Further actions includeproviding the verification phrase as a response to the request for theverification phrase for verifying the identity of the user.

Other versions include corresponding systems, apparatus, and computerprograms, configured to perform the actions of the methods, encoded oncomputer storage devices.

These and other versions may each optionally include one or more of thefollowing features. For instance, in some implementations identifyingsubwords to be included in the verification phrase includes identifyingcandidate subwords, for which stored acoustic data is associated withthe user, as one or more of the subwords to be included in theverification phrase.

In certain aspects, obtaining a candidate phrase that includes at leastsome of the identified subwords as the verification phrase includesdetermining that a particular identified subword is particularly sounddiscriminative and in response to determining that the particularidentified subword is particularly sound discriminative, obtaining acandidate phrase that includes the particular identified subword that isdetermined to be particularly sound discriminative.

In some aspects, obtaining a candidate phrase that includes at leastsome of the identified subwords as the verification phrase includesobtaining multiple candidate phrases including the candidate thatincludes at least some of the identified subwords, determining that thecandidate phrase includes at least some of the identified subwords, andin response to determining that the candidate phrase includes at leastsome of the identified subwords, selecting the determined candidatephrase as the candidate phrase that includes at least some of theidentified subwords from among the multiple candidate phrases.

In some implementations, actions include obtaining acoustic datarepresenting the user speaking the verification phrase, determining thatthe obtained acoustic data matches stored acoustic data for the user,and in response to determining that the obtained acoustic data matchesstored acoustic data for the user, classifying the user as the user.

In certain aspects, determining that the obtained acoustic data matchesstored acoustic data for the user includes determining that storedacoustic data for the at least some of the identified subwords in theverification phrase match obtained acoustic data that correspond to theat least some of the identified subwords in the verification phrase.

In some aspects, identifying subwords to be included in the verificationphrase includes identifying candidate subwords, for which no storedacoustic data is associated with the user, as one or more of thesubwords to be included in the verification phrase. Obtaining acandidate phrase that includes at least some of the identified subwordsas the verification phrase includes obtaining a candidate phrase thatincludes at least one candidate subword for which stored acoustic datais associated with the user and at least one candidate subword for whichno stored acoustic data is associated with the user.

In some implementations, actions include storing acoustic data from theobtained acoustic data that corresponds to the identified candidatesubwords, for which no stored acoustic data is associated with the user,in association with the user.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other potential features, aspects,and advantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a flowchart of an example process for verifying an identity ofa speaker.

FIG. 2 is a block diagram of a system for voice verification enrollment.

FIG. 3 is a block diagram of a system for obtaining a verificationphrase.

FIG. 4 is a block diagram of a system for verifying an identity of aspeaker.

FIG. 5 is another flowchart of an example process for verifying anidentity of a speaker.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

FIG. 1 is a flowchart of an example process 100 for verifying anidentity of a speaker. Generally, the process 100 may include a voiceverification enrollment stage (110). For example, a system may prompt aparticular user to speak an enrollment phrase and store trainingacoustic data representing the particular user speaking the enrollmentphrase. Acoustic data for each of the subwords may be, for example, MFCCcoefficients or filterbank energies representing the particular userspeaking each of the subwords. Subwords may be a phoneme or a sequenceof two or more phonemes, e.g., a triphone. The voice verificationenrollment stage is exemplified in FIG. 2.

The process 100 may include a dynamic generation of a verificationphrase stage (120). For example, in response to a request for averification phrase, the system may dynamically generate a verificationphrase for verifying an identity of a speaker. The dynamic generation ofa verification phrase stage is exemplified in FIG. 3.

The process 100 may include a verification stage (130). For example, thesystem may receive acoustic data representing a speaker speaking theverification phrase and verify the speaker as the particular user basedon the obtained acoustic data. The verification stage is exemplified inFIG. 4.

The process 100 may include a data collection stage (140). For example,after verifying the speaker as the particular user, the system may storethe obtained acoustic data as acoustic data representing the particularuser speaking. The data collection stage is exemplified in FIG. 4.

FIG. 2 is a block diagram of a system 200 for voice verificationenrollment. The system may include an enrollment interface 210 and anacoustic data database 230 for a particular user 220. The system 200 maybe a computing device 212, e.g., a mobile phone. The enrollmentinterface 210 may prompt the particular user 220 to speak an enrollmentphrase to enroll the particular user 220 for voice verification. Forexample, the enrollment interface 210 may prompt the particular user 220to speak the predefined enrollment phrase “DONUT.”

The system 200 may obtain training acoustic data based on the particularuser's 220 speech. The system 200 may obtain the training acoustic databy, for example, performing dynamic time warping to align portions ofthe speech from the particular user 220 with subwords in the enrollmentphrase. For example, the system 200 may align a first portion of speechfrom the particular user 220 with the subword “DO” and a second portionof speech from the particular user 220 with the subword “NUT.”

The system 200 may store, in the acoustic data database 230, trainingacoustic data representing the particular user 220 speaking each of thesubwords in the enrollment phrase. For example, the system 200 may storetraining acoustic data representing the particular user 220 speaking thesubwords “DO” and “NUT” in the enrollment phrase “DONUT.”

For enrollment purposes, the system 200 may use one or more differentenrollment phrases. For example, the system 200 may prompt theparticular user 220 to speak the enrollment phrase, “THE QUICK BROWN FOXJUMPS OVER THE LAZY DOG” and then prompt the particular user 220 to say“COMPUTER PLEASE LEARN MY VOICE FROM THIS PHRASE I'M CURRENTLYSPEAKING.”

The system 200 may use predetermined enrollment phrases, or dynamicallygenerated enrollment phrases. For example, the system 200 may alwaysinitially prompt a user associated with a given locale or language tospeak the enrollment phrase “DONUT,” followed by a predeterminedsequence of additional enrollment terms. Additionally or alternatively,the system 200 may dynamically generate one or more enrollment phrasesthat supplement stored acoustic data. For example, the system 200 mayidentify candidate enrollment phrases that include subwords for whichthe system 200 does not have stored training acoustic data representingthe particular user 220 speaking the subwords.

The system 200 may continue prompting the particular user 220 tocontinue speaking different enrollment phrases until the system 200 hastraining acoustic data representing the particular user 220 speaking atleast a minimum threshold amount of subwords a minimum threshold numberof times. For example, the system 200 may continue prompting theparticular user 220 to continue speaking different enrollment phrasesuntil the system 200 has training acoustic data representing theparticular user 220 speaking at least ten different subwords at leasttwo times each. Additionally or alternatively, the system 200 maycontinue prompting the particular user 220 to continue speaking aparticular enrollment phrase until the system 200 has enough trainingacoustic data for the particular enrollment phrase to satisfy athreshold utterance quality.

In some implementations, the system 200 may also store, in the acousticdata database 230, training acoustic data that the system 200 did notobtain in response to an enrollment phrase. For example, the system 200may identify training acoustic data representing the particular user 220speaking voice commands or voice queries, and store the identifiedtraining acoustic data in the acoustic data database 230.

FIG. 3 is a block diagram of a system 300 for obtaining a verificationphrase. The system 300 may include a speaker verification initiator 304,a subword identifier 310, an acoustic data database 230, a verificationphrase obtainer 320, a candidate phrases database 330, and averification interface 340.

The speaker verification initiator 304, which may be a hotword detector,may receive a voice command and initiate speaker verification based onthe voice command. For example, the speaker verification initiator 304may receive the voice command “OK COMPUTER, UNLOCK,” determine that thevoice command involves speaker verification, and initiate speakerverification. The speaker verification initiator 304 may initiatespeaker verification by providing a speaker verification request to thesubword identifier 310.

In response to a request for speaker verification, the subwordidentifier 310 may identify subwords to be included in a verificationphrase. For example, in response to receiving a request for speakerverification from the speaker verification initiator 304, the subwordidentifier 310 may identify that the subword “NUT” should be included ina verification phrase.

The subword identifier 310 may identify particular subwords to beincluded in the verification phrase based on the training acoustic datastored in the acoustic data database 230. For example, the subwordidentifier 310 may identify the subwords “DO” and “NUT” to be includedin a verification phrase based on determining that the acoustic datadatabase 230 has stored training acoustic data representing theparticular user 200 speaking the subword “DO” and the subword “NUT.”

The verification phrase obtainer 320 may additionally or alternativelyidentify subwords to be included in the verification phrase based ondetermining subwords for which training acoustic data representing theparticular user 220 speaking the subword is not stored in the acousticdata database 230. For example, the verification phrase obtainer 320 mayidentify the subword “PEA” to be included in the verification phrasebased on determining there is little or no stored training acoustic datarepresenting the particular user 220 speaking the subword “PEA.”

The verification phrase obtainer 320 may obtain a verification phrasebased on the identified subwords. For example, the verification phraseobtainer 320 may obtain the verification phrase “PEANUT” based on theidentified subwords “DO,” “NUT,” and “PEA.” The verification phraseobtainer 320 may obtain the verification phrase based on obtainingmultiple candidate phrases from a candidate phrases database 330,identifying an obtained candidate phrase that includes one or more ofthe identified subwords, and selecting the identified candidate phraseas the verification phrase. For example, the verification phraseobtainer 320 may obtain candidates phrases, “KITE,” “BEAR,” “PEANUT,”and “DONUT” from the candidate phrases database 330, identify that thecandidate phrase “PEANUT” includes the identified subwords “NUT” and“PEA,” and select the identified candidate phrase “PEANUT” as theverification phrase.

The verification phrase obtainer 320 may additionally or alternativelyobtain a verification phrase based on which identified subwords havetraining acoustic data for the particular user 220 and which identifiedsubwords do not have training acoustic data for the particular user 220.The verification phrase obtainer 320 may obtain a verification phrasethat has both at least one identified subword that has training acousticdata and at least one identified subword that does not have trainingacoustic data. For example, the verification phrase obtainer 320 mayselect the candidate phrase “PEANUT” as the verification phrase based ondetermining that the candidate phrase “PEANUT” includes the identifiedsubword “NUT” that has training acoustic data and includes theidentified subword “PEA” that does not have training acoustic data.

In some implementations, the verification phrase obtainer 320 may obtaina verification phrase from the candidate phrases based on determining acandidate phrase includes a minimum threshold amount of subwords and apercentage of the subwords in the candidate phrase are subwords thathave training acoustic data for the particular user 220. For example,the verification phrase obtainer 320 may select a candidate phrase “IATE SIX PEANUT BUTTER SANDWICHES TODAY” as a verification phrase basedon determining that the candidate phrase includes at least ten subwordsand approximately 90% of the subwords in the candidate phrase aresubwords that have training acoustic data for the particular user 220.

In selecting a candidate phrase as a verification phrase, theverification phrase obtainer 320 may order a list of obtained candidatephrases by the number of subwords in each candidate phrase for whichtraining acoustic data is stored. From the ordered list, theverification phrase obtainer 320 may select a candidate phrase that hasa minimum threshold number of subwords and a minimum percentage ofsubwords that have training acoustic data for the particular user 220.

In some implementations, the verification phrase obtainer 320 may obtaina verification phrase based on an indication of sound discriminativenessof identified subwords. The verification phrase obtainer 320 maygenerally select a candidate phrase including subwords that are moresound discriminative. The verification phrase obtainer 320 may determinean indication of a sound discriminativeness of each identified subwordand obtain a verification phrase based on selecting a candidate phrasethat includes (i) at least one identified subword that is a subword thatis particularly sound discriminative and has stored acoustic data forthe particular user 220 and (ii) at least one identified subword that isa subword that is particularly sound discriminative and does not havestored acoustic data for the particular user 220. For example, theverification phrase obtainer 320 may select the candidate phrase“PEANUT” as the verification phrase based on determining that thecandidate phrase “PEANUT” includes an identified subword “NUT” that hasstored acoustic data and is particularly sound discriminative, andincludes an identified subword “PEA” that does not have stored acousticdata and is also particularly sound discriminative.

In some implementations, the verification phrase obtainer 320 may obtaina candidate phrase without a candidate phrases database 330. Forexample, the verification phrase obtainer 320 may generate a candidatephrase “NUT PEA” as the verification phrase based on generating acandidate phrase that includes (i) an identified subword “NUT” that hasstored acoustic data and is particularly sound discriminative and (ii)an identified subword “PEA” that does not have stored acoustic data andis also particularly sound discriminative.

The verification interface 340 may prompt the speaker 302 to speak theverification phrase. For example, the verification interface 340 mayoutput on a display of a mobile computing device 202, “PLEASE SAY‘PEANUT.’” Additionally or alternatively, the verification interface 340may output synthesized speech of “PLEASE SAY ‘PEANUT.’”

FIG. 4 is a block diagram of a system 400 for verifying an identity of aspeaker 302. The system 400 may include a subword comparer 420, aspeaker classifier 430, and a welcome interface 440.

The system 400 may obtain acoustic data 410 based on the speaker'sspeech. The system 400 may obtain acoustic data by performing dynamictime warping to align portions of the speech from the speaker 302 withsubwords in the verification phrase. For example, the system 400 mayalign a first portion of speech from the speaker 302 with the subword“PEA” and a second portion of speech from the speaker 302 with thesubword “NUT.” If the system 400 is unable to obtain acoustic data 410for the verification phrase from the speaker's speech, the system 400may generate an error. For example, the system 400 may be unable toalign a verification phrase, “PEANUT,” if the speaker 302 speaks acompletely differently phrase “AARDVARK” and may generate an errorasking the speaker to repeat the verification phrase.

The subword comparer 420 may receive the obtained acoustic data 410representing the speaker 302 speaking one or more subwords of averification phrase. For example, the subword comparer 420 may receiveobtained acoustic data 410 representing the speaker 302 speaking thesubwords “PEA” and “NUT” of the verification phrase “PEANUT.”

The subword comparer 420 may compare the obtained acoustic data 410 withstored training acoustic data in the acoustic data database 230representing the particular user 220 speaking the subwords. For example,the subword comparer 420 may determine a distance between the obtainedacoustic data 410 representing the speaker 420 speaking the subword“NUT” and the stored training acoustic data representing the particularuser 220 speaking the subword “NUT.”

The subword comparer 420 may compare only obtained acoustic data forsubwords that have training acoustic data for the subwords. For example,the subword comparer 420 may determine to compare the obtained acousticdata for the subword “NUT” based on determining that there is storedtraining acoustic data for the subword “NUT.” In another example, thesubword comparer 420 may determine not to compare the obtained acousticdata for the subword “PEA” based on determining that there is no storedtraining acoustic data for the subword “PEA.”

Additionally or alternatively, the subword comparer 420 may compareobtained acoustic data for subwords that do not have training acousticdata with non-user specific acoustic data to verify that the correctsubword was spoken. For example, the subword comparer 420 may compareobtained acoustic data for the subword “PEA” with non-user specificacoustic data for the subword “PEA” to verify that the subword “PEA” wasspoken. In some implementations, the subword comparer 420 may compareobtained acoustic data for subwords that do not have training acousticdata with stored training acoustic data for similar sounding subwords.For example, the subword comparer 420 may compare the obtained acousticdata for the subword “PEA” with stored training acoustic data for asubword “PE.”

The subword comparer 420 may generate a match score for each comparedsubword based on the one or more comparisons of the obtained acousticdata and the stored training acoustic data. The match score may indicatethe likelihood that the particular user 220 spoke the subwordcorresponding to the obtained acoustic data. For example, the subwordcomparer 420 may determine a match score of 90% that indicates 90%likelihood of an identity of the speaker 302 of the subword “PEA” as theparticular user 220 and a match score of 100% that indicates 100%likelihood of an identity of the speaker 302 of the subword “DO” as theparticular user 220.

The subword comparer 420 may generate the match score for each comparedsubword based on determining a distance between the acoustic data andthe stored training acoustic data for the compared subword. The subwordcomparer 420 may determine the distance for each subword based oncomputing L2 distances or performing dynamic time warping matching. Insome implementations when the subword comparer 420 may compare obtainedacoustic data for subwords that do not have training acoustic data withstored training acoustic data for similar sounding subwords, the speakerclassifier 430 may make the comparison more lenient. For example, whenthe subword comparer 420 compares obtained acoustic data for the subword“PEA” with training acoustic data for the subword “PE,” the subwordcomparer 420 may halve any distances.

The subword comparer 420 may generate a final score based on the matchscores. The subword comparer 420 may generate the final score byaveraging the match scores. For example, the subword comparer 420 maygenerate a final score of 95% based on averaging a 90% match score forthe subword “NUT” and a 100% match score for the subword “DO.”

In some implementations, the subword comparer 420 may weight comparisonsbetween obtained acoustic data and stored training acoustic datadifferently for particular subwords. The subword comparer 420 mayprovide greater weight to comparisons for subwords that are determinedto be more sound discriminative or subwords for which more storedacoustic data is available. For example, the subword comparer 420 maydetermine that the subword “NUT” is more sound discriminative than thesubword “DO” and weight the match score of 100% for the subword “NUT”twice as much so that the final score is 97%. In some implementations,the subword comparer 420 may provide the match scores to the speakerclassifier 430 for the speaker classifier to generate a final score.

The speaker classifier 430 may make a classification if the speaker 302is the particular user 220 based on determining that the obtainedacoustic data matches the stored training acoustic data. For example,the speaker classifier 430 may make a classification that the speaker302 is the particular user 220 based on determining the obtainedacoustic data matches the stored training acoustic data because a finalscore from the subword comparer 420 is 90% or greater. In anotherexample, the speaker classifier 430 may make a classification that thespeaker 302 is not the particular user 220 based on determining theobtained acoustic data does not match the stored training acoustic databecause a final score from the subword comparer 420 is less than 90%.

If the speaker classifier 430 makes the classification that the speaker302 is not the particular user 220, another verification phrase may berequested, and the speaker 302 may be prompted to speak the verificationphrase. For example, a locked mobile device may remain locked and mayprompt the speaker 302, “SORRY VOICE NOT RECOGNIZED, PLEASE TRY SPEAKINGINSTEAD ‘CHESTNUT.’” Additionally or alternatively, if the speakerclassifier 430 makes the classification that the speaker 302 is not theparticular user 220, the same verification phrase may be requested. Forexample, a locked mobile device may remain locked and may prompt thespeaker 302, “SORRY VOICE NOT RECOGNIZED, PLEASE TRY REPEATING ‘PEANUT.”In some implementations, the speaker classifier 430 may prompt thespeaker 302 for a particular phrase a pre-determined number, e.g., two,three, or four, of times.

If the speaker classifier 430 makes the classification that the speaker302 is the particular user 220, the speaker classifier 430 may add theobtained acoustic data to the acoustic data database 230 as trainingacoustic data. For example, the speaker classifier 430 may store theobtained acoustic data for the subword “NUT” as representing a secondinstance of the particular user 220 speaking the subword “NUT” and storethe obtained acoustic data for the subword “PEA” as representing a firstinstance of the particular user 220 speaking the subword “PEA.” Inadding the obtained acoustic data to the acoustic data database 230, thespeaker classifier 430 may average the obtained acoustic data for aparticular subword. For example, the speaker classifier 430 may averageacoustic data for two instances of the particular user 220 speaking thesubword “NUT.” By adding the obtained acoustic data to the acoustic datadatabase 230 as acoustic training data, the speaker classifier 430 maymake future comparisons of obtained acoustic data and stored trainingacoustic data for subwords, e.g., “NUT,” more accurate, and enableadditional subwords to be compared in the future as the subwords thatinitially did not have stored training acoustic data, e.g., “PEA,” maynow have stored training acoustic data.

If the speaker classifier 430 makes the classification that the speaker302 is the particular user 220, the speaker classifier 430 mayadditionally or alternatively display the welcome interface 430. Forexample, the welcome interface 430 may be an interface that is initiallydisplayed on the mobile device 202 after the mobile device 202 isunlocked.

Different configurations of the systems 200, 300, and 400 may be usedwhere functionality of the enrollment interface 210, acoustic datadatabase 230, speaker verification initiator 304, subword identifier310, verification phrase obtainer 320, verification interface 340,subword comparer 420, speaker classifier 430, and welcome interface 440may be combined, further separated, distributed, or interchanged. Thesystems 200, 300, and 400 may be implemented in a single device, e.g., amobile device, or distributed across multiple devices, e.g., a clientdevice and a server device.

FIG. 5 is a flowchart of an example process 500 for verifying anidentity of a speaker 302. The following describes the processing 500 asbeing performed by components of the systems 300 and 400 that aredescribed with reference to FIGS. 3 and 4. However, the process 500 maybe performed by other systems or system configurations.

The process 500 may include receiving a request for a verificationphrase for verifying the identity of the speaker 302 (510). For example,the system 200 may receive a voice command “OK COMPUTER, UNLOCK” fromthe speaker 302 to unlock the mobile device 202, the speakerverification initiator 304 may provide a request for the verificationphrase to the subword identifier 310, and the subword identifier 310 mayreceive the request for the verification phrase.

The process 500 may include identifying subwords to be included in theverification phrase (520). For example, in response to receiving therequest for the verification phrase for verifying the identity of thespeaker 302, the subword identifier 310 may identify that the subwords“I,” “WANT,” “TO,” “BE,” “AN,” “ASTRO,” “NAUT,” “WHEN,” “GROW,” and“UP,” should be included in a verification phrase based on determiningthat training acoustic data representing the particular user 220speaking the subwords “I,” “WANT,” “TO,” “BE,” “AN,” “NAUT,” “WHEN,”“GROW,” and “UP,” is stored in the acoustic data database 230 anddetermining that the subword “ASTRO” is particularly sounddiscriminative and that no training acoustic data representing theparticular user 220 speaking the subword “ASTRO” is stored in theacoustic data database 230.

The process 500 may include obtaining a candidate phrase that includesat least some of the identified subwords as the verification phrase(530). For example, in response to identifying the subwords to beincluded in the verification phrase, the verification phrase obtainer320 may obtain multiple candidate phrases “I WANT TO BE AT GREAT FALLSPARK TODAY” and “I WANT TO BE AN ASTRONAUT WHEN I GROW UP” from acandidate phrases database 330, determine that the particular candidatephrase, “I WANT TO BE AN ASTRONAUT WHEN I GROW UP,” (i) includes atleast ten subwords, (ii) at least 90% of the subwords have storedtraining acoustic data, and (iii) includes the identified subword“ASTRO” that is particularly sound discriminative and for which there isno stored training acoustic data, and based on the determination, selectthe particular candidate phrase as the verification phrase.

The process 500 may include providing the verification phrase as aresponse to the request for the verification phrase for verifying theidentity of the speaker 302 (540). For example, the verificationinterface 340 may display “PLEASE SAY ‘I WANT TO BE AN ASTRONAUT WHEN IGROW UP.’”

The process 500 may include obtaining acoustic data representing thespeaker 302 speaking the subwords of the verification phrase (550). Forexample, the subword comparer 420 may obtain acoustic data representingthe speaker 302 speaking each subword in the candidate phrase “I WANT TOBE AN ASTRONAUT WHEN I GROW UP.”

The process 500 may include determining that the obtained acoustic datamatches stored training acoustic data (560). For example, the subwordcomparer 420 may generate a final score of 90% based on averaging matchscores for each of the subwords “I,” “WANT,” “TO,” “BE,” “AN,” “NAUT,”“WHEN,” “GROW,” and “UP,” where the match scores are determined based oncalculating a distance for each of the subwords between the obtainedacoustic data and the stored training acoustic data.

The process 500 may include classifying the speaker 302 as theparticular user 220 (570). For example, the speaker classifier 430 maydetermine that a final score of 90% generated by the subword comparer420 is at least 90%. In response to classifying the speaker 302 as theparticular user 220, the speaker classifier 403 may store the obtainedacoustic data in the acoustic data database 230 as training acousticdata and trigger a display of a welcome interface 440.

In some implementations, principles of the process 500 may also be usedfor speaker identification. For example, the system 400 may compare theobtained acoustic data to stored training acoustic data for multipleusers to generate final scores for each of the multiple users, determinethat the final score for the particular user is the only final scorethat is at least 90%, and identify the speaker as the particular user.

Embodiments of the subject matter, the functional operations and theprocesses described in this specification can be implemented in digitalelectronic circuitry, in tangibly-embodied computer software orfirmware, in computer hardware, including the structures disclosed inthis specification and their structural equivalents, or in combinationsof one or more of them. Embodiments of the subject matter described inthis specification can be implemented as one or more computer programs,i.e., one or more modules of computer program instructions encoded on atangible nonvolatile program carrier for execution by, or to control theoperation of, data processing apparatus. Alternatively or in addition,the program instructions can be encoded on an artificially generatedpropagated signal, e.g., a machine-generated electrical, optical, orelectromagnetic signal that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus. The computer storage medium can be amachine-readable storage device, a machine-readable storage substrate, arandom or serial access memory device, or a combination of one or moreof them.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, or multiple processors or computers.The apparatus can include special purpose logic circuitry, e.g., an FPGA(field programmable gate array) or an ASIC (application specificintegrated circuit). The apparatus can also include, in addition tohardware, code that creates an execution environment for the computerprogram in question, e.g., code that constitutes processor firmware, aprotocol stack, a database management system, an operating system, or acombination of one or more of them.

A computer program (which may also be referred to or described as aprogram, software, a software application, a module, a software module,a script, or code) can be written in any form of programming language,including compiled or interpreted languages, or declarative orprocedural languages, and it can be deployed in any form, including as astandalone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A computer program may, butneed not, correspond to a file in a file system. A program can be storedin a portion of a file that holds other programs or data (e.g., one ormore scripts stored in a markup language document), in a single filededicated to the program in question, or in multiple coordinated files(e.g., files that store one or more modules, sub programs, or portionsof code). A computer program can be deployed to be executed on onecomputer or on multiple computers that are located at one site ordistributed across multiple sites and interconnected by a communicationnetwork.

The processes and logic flows described in this specification can beperformed by one or more programmable computers executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Computers suitable for the execution of a computer program include, byway of example, can be based on general or special purposemicroprocessors or both, or any other kind of central processing unit.Generally, a central processing unit will receive instructions and datafrom a read-only memory or a random access memory or both. The essentialelements of a computer are a central processing unit for performing orexecuting instructions and one or more memory devices for storinginstructions and data. Generally, a computer will also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto optical disks, or optical disks. However, a computer need nothave such devices. Moreover, a computer can be embedded in anotherdevice, e.g., a mobile telephone, a personal digital assistant (PDA), amobile audio or video player, a game console, a Global PositioningSystem (GPS) receiver, or a portable storage device (e.g., a universalserial bus (USB) flash drive), to name just a few.

Computer readable media suitable for storing computer programinstructions and data include all forms of nonvolatile memory, media andmemory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto optical disks; andCD-ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending web pages to a web browser on auser's client device in response to requests received from the webbrowser.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front end component, e.g., aclient computer having a graphical user interface or a Web browserthrough which a user can interact with an implementation of the subjectmatter described in this specification, or any combination of one ormore such back end, middleware, or front end components. The componentsof the system can be interconnected by any form or medium of digitaldata communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of what may beclaimed, but rather as descriptions of features that may be specific toparticular embodiments. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous. Other steps may be provided, or stepsmay be eliminated, from the described processes. Accordingly, otherimplementations are within the scope of the following claims.

1. (canceled)
 2. A computer-implemented method, comprising: providing a speaker identification verification phrase; obtaining audio data representing a candidate user speaking the speaker identification verification phrase; obtaining, for each of multiple subwords associated with the speaker identification phrase, sample acoustic features that are derived from the audio data representing the candidate user speaking the speaker identification verification phrase; obtaining, for each of the multiple subwords associated with the speaker identification verification phrase, reference acoustic features that (i) are stored in a collection of acoustic features for a target user, and (ii) are derived from audio data of the target user speaking one or more words that include the subword; determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features; in response to determining that the sample acoustic features are associated with the reference acoustic features, identifying the candidate user as the target user; obtaining, for each of one or more other subwords associated with speaker identification verification phrase, other acoustic features that (i) are not already stored in the collection of acoustic features for the target user, and (ii) are derived from the audio data representing the candidate user speaking the speaker identification verification phrase; and in response to identifying the candidate user as the target user, storing one or more of the other acoustic features in the collection of acoustic features for the target user.
 3. The method of claim 2, wherein the speaker identification verification phrase includes the multiple subwords and the one or more other subwords.
 4. The method of claim 2, comprising: in response to identifying the candidate user as the target user, updating the collection of acoustic features for the target user with the one or more of the acoustic features.
 5. The method of claim 4, wherein updating the collection of acoustic features for the target user with the one or more of the acoustic features comprises: storing the one or more of the acoustic features in the collection of acoustic features for the target user.
 6. The method of claim 2, wherein determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features, comprises: determining, for each of the multiple subwords associated with the speaker identification verification phrase, a distance between the sample acoustic features and the reference acoustic features; and based on at least the distance, determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features.
 7. The method of claim 6, wherein based on at least the distance, determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features, comprises: determining sound discriminativeness of the subword; and based on at least the distance and the sound discriminativeness, determining that the sample acoustic features are associated with the reference acoustic features.
 8. The method of claim 2, comprising: determining, for each of the one or more other subwords associated with the speaker identification verification phrase, that the sample acoustic features are not associated with the reference acoustic features.
 9. A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: providing a speaker identification verification phrase; obtaining audio data representing a candidate user speaking the speaker identification verification phrase; obtaining, for each of multiple subwords associated with the speaker identification phrase, sample acoustic features that are derived from the audio data representing the candidate user speaking the speaker identification verification phrase; obtaining, for each of the multiple subwords associated with the speaker identification verification phrase, reference acoustic features that (i) are stored in a collection of acoustic features for a target user, and (ii) are derived from audio data of the target user speaking one or more words that include the subword; determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features; in response to determining that the sample acoustic features are associated with the reference acoustic features, identifying the candidate user as the target user; obtaining, for each of one or more other subwords associated with speaker identification verification phrase, other acoustic features that (i) are not already stored in the collection of acoustic features for the target user, and (ii) are derived from the audio data representing the candidate user speaking the speaker identification verification phrase; and in response to identifying the candidate user as the target user, storing one or more of the other acoustic features in the collection of acoustic features for the target user.
 10. The system of claim 9, wherein the speaker identification verification phrase includes the multiple subwords and the one or more other subwords.
 11. The system of claim 9, the operations comprising: in response to identifying the candidate user as the target user, updating the collection of acoustic features for the target user with the one or more of the acoustic features.
 12. The system of claim 11, wherein updating the collection of acoustic features for the target user with the one or more of the acoustic features comprises: storing the one or more of the acoustic features in the collection of acoustic features for the target user.
 13. The system of claim 9, wherein determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features, comprises: determining, for each of the multiple subwords associated with the speaker identification verification phrase, a distance between the sample acoustic features and the reference acoustic features; and based on at least the distance, determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features.
 14. The system of claim 13, wherein based on at least the distance, determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features, comprises: determining sound discriminativeness of the subword; and based on at least the distance and the sound discriminativeness, determining that the sample acoustic features are associated with the reference acoustic features.
 15. The system of claim 9, the operations comprising: determining, for each of the one or more other subwords associated with the speaker identification verification phrase, that the sample acoustic features are not associated with the reference acoustic features.
 16. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising: providing a speaker identification verification phrase; obtaining audio data representing a candidate user speaking the speaker identification verification phrase; obtaining, for each of multiple subwords associated with the speaker identification phrase, sample acoustic features that are derived from the audio data representing the candidate user speaking the speaker identification verification phrase; obtaining, for each of the multiple subwords associated with the speaker identification verification phrase, reference acoustic features that (i) are stored in a collection of acoustic features for a target user, and (ii) are derived from audio data of the target user speaking one or more words that include the subword; determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features; in response to determining that the sample acoustic features are associated with the reference acoustic features, identifying the candidate user as the target user; obtaining, for each of one or more other subwords associated with speaker identification verification phrase, other acoustic features that (i) are not already stored in the collection of acoustic features for the target user, and (ii) are derived from the audio data representing the candidate user speaking the speaker identification verification phrase; and in response to identifying the candidate user as the target user, storing one or more of the other acoustic features in the collection of acoustic features for the target user.
 17. The medium of claim 16, wherein the speaker identification verification phrase includes the multiple subwords and the one or more other subwords.
 18. The medium of claim 16, the operations comprising: in response to identifying the candidate user as the target user, updating the collection of acoustic features for the target user with the one or more of the acoustic features.
 19. The medium of claim 18, wherein updating the collection of acoustic features for the target user with the one or more of the acoustic features comprises: storing the one or more of the acoustic features in the collection of acoustic features for the target user.
 20. The medium of claim 16, wherein determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features, comprises: determining, for each of the multiple subwords associated with the speaker identification verification phrase, a distance between the sample acoustic features and the reference acoustic features; and based on at least the distance, determining, for each of the multiple subwords associated with the speaker identification verification phrase, that the sample acoustic features are associated with the reference acoustic features.
 21. The medium of claim 16, the operations comprising: determining, for each of the one or more other subwords associated with the speaker identification verification phrase, that the sample acoustic features are not associated with the reference acoustic features. 